Optimally Sparse Representation in General (non-Orthogonal) Dictionaries via 1 Minimization

Given a ‘dictionary’ D = {dk} of vectors dk, we seek to represent a signal S as a linear combination S = ∑ k γ(k)dk, with scalar coefficients γ(k). In particular, we aim for the sparsest representation possible. In general, this requires a combinatorial optimization process. Previous work considered the special case where D is an overcomplete system consisting of exactly two orthobases, and has shown that, under a condition of mutual incoherence of the two bases, and assuming that S has a sufficiently sparse representation, this representation is unique and can be found by solving a convex optimization problem: specifically, minimizing the 1 norm of the coefficients γ. In this paper, we obtain parallel results in a more general setting, where the dictionary D can arise from two or several bases, frames, or even less structured systems. We introduce the Spark, a measure of linear dependence in such a system; it is the size of the smallest linearly dependent subset (dk). We show that, when the signal S has a representation using less than Spark(D)/2 nonzeros, this representation is necessarily unique. We develop bounds on the Spark that reproduce uniqueness results given in special cases considered earlier. We also show that in the general dictionary case, any sufficiently sparse representation of S is also the unique minimal 1-representation, using sparsity thresholds related to our Spark bounds. We sketch three applications: separating linear features from planar ones in 3-D data, noncooperative multiuser encoding, and identification of overcomplete independent components models.